Learning from Digital Library Evaluations

In this paper we analyse evaluation studies of the Europeana digital library from its launch in 2009 until today. Using Saracevic’s digital library evaluation framework, the studies are categorised by their constructs, contexts, criteria, and methodologies. Concentrating on studies that evaluate Europeana services or single components, we show gaps in the evaluation of certain Europeana aspects. Finally, we derive strategies for building an evaluation archive that serves as memory and supports comparisons.Im vorliegenden Artikel analysieren wir Evaluationsstudien der digitalen Bibliothek Europeana von 2009 bis heute. Unter Berücksichtigung von Saracevic’ Evaluationsframework für digitale Bibliotheken werden die Studien nach ihren Konstrukten, Kontexten, Kriterien und Methodologien kategorisiert. Die Analyse konzentriert sich auf Studien, die Dienstleistungen oder einzelne Komponenten von Europeana evaluieren, und zeigt Lücken in der Evaluation bestimmter Aspekte von Europeana auf. Schließlich werden Strategien diskutiert, um ein Evaluationsarchiv zu entwickeln, welches sowohl der Langzeitarchivierung dient als auch Vergleiche von Evaluationsergebnissen unterstützt.Peer Reviewe

Interface Language, User Language and Success Rates in The European Library

In this paper, TEL 2010 action logs are analyzed with a particular focus on the impact of language (user native language and interface language) on the success of a search session. Particular user actions are defined as success indicators for searches and sessions are divided into “successful” and “unsuccessful” sessions with respect to their outcomes. Two approaches for studying the impact of the language of the search interface are pursued: (1) the effect of concurrent language choice when associating the user language (determined by IP address) with the interface language and (2) the consequences of interface language changes during a session. The challenges of country and language identification via IP addresses are also discussed

Disinformation in the context of health information behaviour

During the Covid-19-pandemic, health information behaviour received much attention and was put into focus, especially as much disinformation around the topic spread on social media and in different information sources. Disinformation is misleading, misunderstanding or even completely false information that can harm a person, social group, organisation or even a country and potentially have a dangerous impact (Fallis, 2015). Although the concept of disinformation has been researched in information behaviour before the pandemic (e.g. Karlova and Fisher, 2013), it is increasingly discussed in other research fields and the health domain due to its potential to harm. Disinformation in the health context can lead to poor health choices and can inhibit efforts to relieve health crises, for example mistrust in vaccinations against Covid-19 disease, and impacts health information literacy (e.g. Darrin and Elliot, 2020; De Gani et al., 2022). The need for strategies to counteract the harmful influences of disinformation in the health domain is inevitable. For that, we need to better understand the behaviour that results in disinformation and evolves around it.Peer Reviewe

Initial specification of the evaluation tasks "Use cases to bridge validation and benchmarking" PROMISE Deliverable 2.1

Evaluation of multimedia and multilingual information access systems needs to be performed from a usage oriented perspective. This document outlines use cases from the three use case domains of the PROMISE project and gives some initial pointers to how their respective characteristics can be extrapolated to determine and guide evaluation activities, both with respect to benchmarking and to validation of the usage hypotheses. The use cases will be developed further during the course of the evaluation activities and workshops projected to occur in coming CLEF conferences

Une classification interdisciplinaire des champs de recherche pour la science

Dieser Beitrag ist mit Zustimmung des Rechteinhabers (De Gruyter) frei zugänglich.In einem 2020 durchgeführten Kooperationsprojekt zwischen der Humboldt-Universität zu Berlin und dem Deutschen Zentrum für Hochschul- und Wissenschaftsforschung (DZWH) ist eine Forschungsfeldklassifikation entstanden, die problembezogene und interdisziplinäre Forschung in Deutschland abbildet. Ziel dieser Klassifikation ist es, Forschungsberichterstattung basierend auf disziplinorientierten Fächersystematiken innerhalb des Kerndatensatz Forschung sinnvoll zu ergänzen und somit der einem ständigen Wandel unterliegenden Forschung Rechnung zu tragen. Die Klassifikation ist in einem intellektuellen Prozess mittels Card-Sorting entstanden und hat vorhandene Forschungsklassifikationen, Forschungsfeldlisten und aktuelle Forschungsprojekte als Grundlage herangezogen. Die umfassende Einbeziehung von Stakeholdern bei der Entwicklung der Klassifikation soll deren Akzeptanz steigern und einen vielfältigen Einsatz ermöglichen.In 2020, a cooperative project at the Humboldt-Universität zu Berlin and the German Centre for Higher Education Research and Science Studies (DZHW) developed a research field classification to categorize problem-related and interdisciplinary research in Germany. The aim of this classification is to improve the reporting of research information by offering a complement to discipline-oriented subject classifications, e.g. in the context of the Research Core Dataset (Kerndatensatz Forschung) – a standard for reporting on research and research activities. The research field classification is sought to reflect the changing research landscape. The classification was intellectually developed by using a card sorting method and considering existing research classifications, field lists and current research projects. The extensive involvement of stakeholders in the development of the classification is intended to increase its acceptance and facilitate use in diverse settings.Dans le cadre d'un projet de coopération mené en 2020 entre l'Université Humboldt de Berlin et le Deutschen Zentrum für Hochschul- und Wissenschaftsforschung (Centre allemand de recherche universitaire et scientifique – DZWH), une classification des domaines de recherche a été élaborée, qui reflète la recherche axée sur des problèmes, ainsi que la recherche interdisciplinaire en Allemagne. L'objectif de cette classification est de compléter judicieusement la présentation des rapports de recherche, qui sont basés sur des classifications de matières orientées vers les disciplines. Cette amélioration de l'ensemble de données de base permet ainsi de tenir compte de l’évolution constante de la recherche. La classification est le résultat d'un processus intellectuel de tri de cartes et s'est appuyée sur des classifications de recherche existantes, des listes de domaines de recherche et des projets de recherche actuels. L'implication large des parties prenantes dans le développement de la classification devrait augmenter son acceptation et permettre des utilisations variées.Peer Reviewe

Wie können interdisziplinäre Forschungsfelder klassifiziert werden?

Peer Reviewe

Cultural Analytics - Evaluating Cultural Heritage Information Systems

With vast amounts of digitized cultural heritage organized and made accessible in large information systems, methods, workflows and techniques need to be developed to evaluate the quality of cultural heritage (CH) information systems. As these systems are very different from traditional information systems due to their heterogeneous content, the lack of best practices and reliable use cases, evaluation methods need to be adapted and targeted to deliver reliable results. The half-day workshop will bring together experts, researchers and teachers from various disciplines to discuss and develop evaluation approaches adapted to cultural heritage information systems. This workshop comprises approaches assessing the data quality, user and task analysis and system performance of these systems aiming for a holistic overview of complementary research questions, methods and workflows. The results should prove to be useful for standardizing evaluation efforts and campaigns in this domain, offering a reusable repertoire of evaluation methods that makes them comparable.ye

First Report on Alternative Evaluation Methodology PROMISE Deliverable 4.1

The first report on alternative evaluation methodology summarizes work done within the PROMISE environment and especially within Work package 4 - Evaluation Metrics and Methodologies. The report outlines efforts to develop and support alternative, automated evaluation methodologies, with a special focus on generating ground truth from existing data sources like Log files or annotations. Events like LogCLEF 2011, PatOlympics 2011 or the CHiC2011 workshop are presented and reviewed on their impact on the three main uses case domains

Agrobacterium rhizogenes-Mediated Transformation of the Parasitic Plant Phtheirospermum japonicum

Background: Plants within the Orobanchaceae are an agriculturally important group of parasites that attack economically important crops to obtain water and nutrients from their hosts. Despite their agricultural importance, molecular mechanisms of the parasitism are poorly understood. Methodology/Principal Findings: We developed transient and stable transformation systems for Phtheirospermum japonicum, a facultative parasitic plant in the Orobanchaceae. The transformation protocol was established by a combination of sonication and acetosyringone treatments using the hairy-root-inducing bacterium, Agrobacterium rhizogenes and young seedlings. Transgenic hairy roots of P. japonicum were obtained from cotyledons 2 to 3 weeks after A. rhizogenes inoculation. The presence and the expression of transgenes in P. japonicum were verified by genomic PCR, Southern blot and RT-PCR methods. Transgenic roots derived from A. rhizogenes-mediated transformation were able to develop haustoria on rice and maize roots. Transgenic roots also formed apparently competent haustoria in response to 2,6dimethoxy-1,4-benzoquinone (DMBQ), a haustorium-inducing chemical. Using this system, we introduced a reporter gene with a Cyclin B1 promoter into P. japonicum, and visualized cell division during haustorium formation. Conclusions: We provide an easy and efficient method for hairy-root transformation of P. japonicum. Transgenic marker analysis revealed that cell divisions during haustorium development occur 24 h after DMBQ treatment. The protocol